Speeding up Arithmetic Coding using Greedy Re-normalization

A typical adaptive arithmetic coder consists of three steps: range calculation, renormalization, and probability model updating. A method, called greedy re-normalization, is given in this paper that significantly reduces the computational complexity of the renormalization step of arithmetic coding. This is achieved by reducing both the number of re-normalizations required to encode a sequence and the number of operations within each re-normalization. Following the notations in [1], the internal state of an arithmetic coder is represented by an interval [L,L+R) where L is the base of the interval and R the length, and both L and R are b-bit integers. To reduce the number of operations within each re-normalization, the greedy re-normalization method generates as many code bits as possible from an interval and updates the interval correspondingly; moreover, both of the tasks are done at one time without using a loop. This is one of the differences between the greedy re-normalization method and the conventional method in [1] where a loop is used to generate code bits and update an interval. Let m(L,R) represent the maximal number of code bits with known polarities that can be generated from [L,L + R) and k(L,R) the maximum number of outstanding code bits (see [1] for the meaning of this term). We show that m(L,R) = b−1− log t , and k(L,R) = log t − log(2 log t − t) if R = 1 and log(2 log t − t) ≥ logR , and log t − logR otherwise. (log stands for the logarithm with base 2.) The maximal number of bits that can be shifted out of L and R is given by m(L,R) + k(L,R). To reduce the number of re-normalizations in encoding a sequence, the proposed method adopts a dynamic re-normalization criterion: Rpmin < 1, where pmin stands for the probability of the least probable symbol in the source alphabet. Since the value pmin may change from time to time, this criterion is a dynamic one. This is another difference between the proposed method and the conventional method where a fixed re-normalization threshold is usually used. Experimental results show that the proposed greedy re-normalization method indeed improves the speed of arithmetic coding. For example, by simply replacing the re-normalization procedure in the binary arithmetic coder in [1] with the proposed greedy re-normalization, more than 20% gain in coding speed is observed in experiments. By combining the greedy re-normalization method with some ideas of the QM-Coder, about 30% average gain over the QM-coder is observed for binary i.i.d. sources with the probability of the less probable symbol ranging from 0.01 to 0.45. Compression performance is not compromised in either of the two cases. The greedy re-normalization method can also be used in multi-symbol arithmetic coding.